Section: Partnerships and Cooperations

International Initiatives

DALHIS

Participants : Christine Morin, Anne-Cécile Orgerie, Javier Rojas Balderrama, Matthieu Simonin, Cédric Tedeschi.

• Title: Data Analysis on Large Heterogeneous Infrastructures for Science

• International Partner (Institution - Laboratory - Researcher):

  • Lawrence Berkeley National Laboratory, Berkeley, USA

  • Data Science and Technology department

  • French PI: Christine Morin

  • American PI: Deb Agarwal, head of the Data Science and Technology department

• Duration: 2013 - 2015

• See also: https://project.inria.fr/dalhis/

• The worldwide scientific community is generating large datasets at increasing rates causing data analysis to emerge as one of the primary modes of science. Existing data analysis methods, tools and infrastructure are often difficult to use and unable to handle the “data deluge”. A scientific data analysis environment needs to address three key challenges: a) programmability: easily composable and reusable programming environments for analysis algorithms and pipeline execution, b) agility: software that can adapt quickly to changing demands and resources, and, c) scalability: take advantage of all available resource environments including desktops, clusters, grids, clouds and HPC environments. The goal of the DALHIS associated team is to coordinate research and create together a software ecosystem to facilitate data analysis seamlessly across desktops, HPC and cloud environments. Specifically, our end goal is to build a dynamic environment that is user-friendly, scalable, energy-efficient and fault tolerant through coordination of existing projects. We plan to design a programming environment for scientific data analysis workflows that will allow users to easily compose their workflows in a programming environment such as Python and execute them on diverse high-performance computing (HPC) and cloud resources. We will develop an orchestration layer for coordinating resource and application characteristics. The adaptation model will use real-time data mining to support elasticity, fault-tolerance, energy efficiency and provenance. We investigate how to provide execution environments that allow users to seamlessly execute their dynamic data analysis workflows in various research environments. The work done in 2014 on scientific workflows, energy efficiency and data management is described respectively in Sections 6.5.1 , 6.4.1 and 6.3.3 . Christine Morin, Anne-Cécile Orgerie, Javier Rojas Balderrama, Cédric Tedeschi and Deb Agarwal participated in the BIS 2014 workshop held in Paris in June 2014. Christine Morin and Deb Agarwal were the co-chairs of the session on Computation and communication for the Future Internet at BIS 2014. Cédric Tedeschi presented the DALHIS activities during this session focusing on our results on scientific workflows. Deb Agarwal has been awarded an Inria International Chair for the 2015-2019 period enabling long visits in the Myriads team.

Northeastern University

We started a collaboration with Professor Gene Cooperman, Northeastern University, Boston, USA on the design of a cloud agnostic checkpointing service on top of IaaS clouds for reliable application execution, inter-cloud application migration and easing application "cloudification". Gen Cooperman was hosted in Myriads team for a 1.5-month visit in March-April 2014. His PhD student, Jiajun Cao did a 3-month internship in Myriads team from May to August 2014.

ORNL/TTU

We collaborate on cloud computing with Stephen Scott, Professor at Tennessee Tech University (TTU) and researcher at Oak Ridge National Laboratory (ORNL) in the USA. He visited Myriads team in July 2014 to investigate synergetic work directions on cloud security.

Argonne/ Chicago University

We collaborate on cloud computing with Kate Keahey from Argonne National Laboratory, USA. She hosted Ismael Cuadrado Cordero in her team for a 12-week summer internship (June-September 2014) on using extended on-availability leases to increase utilization in scientific IaaS clouds.

University of Guadalajara

Nikolaos Parlavantzas is collaborating with the team of Prof. Héctor Duran-Limon of the University of Guadalajara, Mexico, preparing a joint ANR-CONACYT project submission.

VU University amsterdam

We collaborate with Thilo Kielmann's research group at VU University Amsterdam on research and development around the ConPaaS system. This collaboration has lead to two joint publications this year, and another paper has been accepted in 2015.